1. Field of the Invention
The present invention relates generally to recognition systems and more particularly to trainable or learning systems, which are capable of modifying their own internal processing in response to information descriptive of system performance.
2. Description of the Prior Art
Recognition systems of the type that recognize patterns deal with the problem of relating a set of or sequence of input objects or situations to what is already known by the system. This operation is a necessary part of any intelligent system since such a system must relate its current input and input environment to what it has experienced in order to respond appropriately.
A pattern recognition task is typically divided into three steps: data acquisition, feature extraction, and pattern classification. The data acquisition step is performed by a transducer which converts measurements of the pattern into digital signals appropriate for a recognition system. In the feature extraction step these signals are converted into a set of features or attributes which are useful for discriminating the patterns relevant to the purposes of the recognizer. In the final step of pattern classification these features are matched to the features of the classes known by the system to decide which class best explains the input pattern.
The division between the step of feature extraction and pattern classification is somewhat arbitrary. A powerful feature extractor would make the classifier's job trivial and conversely, a powerful decision mechanism in the classifier would perform well even with simple features. However in practice, feature extractors tend to be more task dependent. For example, data acquisition and feature extraction for hand-printed character recognition will differ from that needed for speech recognition. Pattern classification on the other hand can be designed to be task independent, although it often is not.
A particular category of pattern recognition tasks is characterized by whether or not the features can be reduced to a linear sequence of input objects for the classification step. This category is called sequential pattern recognition. Examples of tasks which naturally fall into this category are optical character recognition, waveform recognition, and speech recognition. Other tasks such as computer image recognition can be placed within sequential pattern recognition by an appropriate ordering of the features.
Patterns of features must be acquired by the pattern recognizer for a new class of features before the system can recognize the class. When patterns cannot be learned from examples, acquisition of the patterns is a major problem.
Prior art optical character and speech recognition systems correlate input patterns with a set of templates, in order to determine a "best match". A correlation is performed using a particular algorithm which is specifically derived for the matching operation required for a particular problem such as speech recognition, character recognition, etc. . . A change in type font or speaker, for example, would require replacing the templates and changing parameters of the alqorithm in such prior art systems.
Many trainable systems exist in the prior art, of which the following U.S. Patents are descriptive. U.S. Pat. No. 3,950,733, an Information Processing System, illustrates an adaptive information processing system in which the learning growth rate is exponential rather than linear. U.S. Pat. No. 3,715,730, a Multi-criteria Search Procedure for Trainable Processors illustrates a system having an expanded search capability in which trained responses to input signals are produced in accordance with predetermined criteria. U.S. Pat. No. 3,702,986, a Trainable Entropy System illustrates a series of trainable non-linear processors in cascade. U.S. Pat. No. 3,700,866, a Synthesized Cascaded Processor System illustrates a system in which a series of trainable processors generate a probabilistic signal for the next processor in the cascade which is a best estimate for that processor of a desired response. U.S. Pat. Nos. 3,638,196 and 3,601,811, Learning Machines, illustrate the addition of hysteresis to perceptron-like systems. U.S. Pat. No. 3,701,974, Learning Circuit, illustrates a typical learning element of the prior art. U.S. Pat. No. 3,613,084, Trainable Digital Apparatus illustrates a deterministic synethesized boolean function. U.S. Pat. No. 3,623,015, Statistical Pattern Recognition System With Continual Update of Acceptance Zone Limits, illustrates a pattern recognition system capable of detecting similarities between patterns on a statistical basis. U.S. Pat. Nos. 3,999,161 and 4,066,999 relate to statistical character recognition systems having learning capabilities.
Other patents that deal with learning systems that appear to be adaptive based upon probability or statistical experience include U.S. Pat. Nos. 3,725,875; 3,576,976; 3,678,461; 3,440,617 and 3,414,885. Patents showing logic circuits that may be used in the above systems include U.S. Pat. Nos. 3,566,359; 3,562,502; 3,446,950; 3,103,648; 3,646,329; 3,753,243; 3,772,658; and 3,934,231.
Adaptive pattern, speech or character recognition systems are shown in the following U.S. Pat. Nos. 4,318,083; 4,189,779; 3,581,281; 3,588,823; 3,196,399; 4,100,370; and 3,457,552. U.S. Pat. No. 3,988,715 describes a system that develops conditional probabilities character by character with the highest probability being selected as the most probable interpretation of an optically scanned word. U.S. Pat. No. 3,267,431 describes a system that uses a "perceptron", a weighted correlation network, that is trained on sample patterns for identification of other patterns.
Articles and publications relating to the subject matter of the invention include the following: Introduction To Artifical Intelligence, P. C. Jackson Jr., Petrocelli/Charter, N. Y. 1974 pages 368-381; "Artifical Intelligence", S. K. Roberts, Byte, Vol. 6, No. 9, September 1981, pages 164-178; "How Artificial Is Intelligence?", W. R. Bennett Jr., American Scientist, Vol. 65, November-December 1977, pages 694-702; and "Machine Intelligence and Communications In Future NASA Missions", T. J. Healy, IEEE Communications Magazine, Vol. 19, No. 6, November 1981, pages 8-15.